Artificial associative neuron synapse

ABSTRACT

The invention relates to a method for determining a weight coefficient of an artificial associative neuron synapse, where the synaptic weight coefficient is determined on the basis of the temporal average of a product of two signals. The method comprises the steps of taking temporal samples from the product of said two signals at such moments when one of the signals starts to deviate from zero, feeding said samples into such a shift register chain, from where a predetermined number of said samples taken at previous moments are continuously available, deducing on the basis of said samples taken at previous moments, whether a value deviating from zero is to be set as the synaptic weight coefficient. The invention also relates to an artificial associative neuron synapse.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is the national stage of International Application PCT/FI01/00950having international filing date of 31 Oct. 2001, published in EnglishJul. 4, 2002 (WO 02/052500 A1), which in turn claims priority fromFinnish Patent Application FI 20002853, filed 22 Dec. 2000.

BACKGROUND OF THE INVENTION

1. Technical Field

The invention relates to a method for determining a weight coefficientfor an artificial associative neuron synapse and a correspondingartificial associative neuron synapse. The invention particularlyrelates to implementing the synapse in such an artificial associativeneuron in which the weight coefficient, or strength, of the synapse isdetermined on the basis of a temporal average of the product of twosignals according to the Hebb rule or a modification thereof (Jain, AnilK., Mao Jianchang, Mohiuddin K. M. (1996) “Artificial Neural Networks: ATutorial” in Computer, March 1996, p. 31 to 44).

2. Discussion of Related Art

International patent application WO 98/43159 discloses an artificialassociative neuron that can be used in artificial neural networks. Theneurons are connected in parallel to form layers, and such layers arearranged in succession to form a neural network. Neural networks can beused in various calculation and pattern recognition tasks.

FIG. 1 illustrates a prior art associative neuron including n synapses.Since the number n may be quite high (ranging from 100 to 10 000, forexample), FIG. 1 shows only the first, second and nth synapse of theneuron. The input signals of the neuron are a main signal s andassociative signals a₁, a₂, . . . a_(n). Label so indicates the outputsignal of the neuron. Said signals are, for example, voltage levels.Circuit blocks 1 to 4, 8 to 11 and 12 to 15 correspondingly describe thefirst, second and nth synapse of the neuron, and its operation isdescribed in greater detail below. Block 5 is a summing circuit in whichthe output signals obtained from the synapses are summed. Block 6 is asumming circuit in which the main signal s and the sum of the outputsignals obtained from the synapses are summed. Block 7 is a thresholdcircuit.

Information about the simultaneous occurrence of the main signal sarriving at the associative neuron and the associative signal a_(i)(where i=1,2 . . . ,n) arriving at the synapse is stored in thesynapses. This information may be presented, for example, in voltageform so that the voltage concerned represents the temporal average ofthe product s*a_(i) of the main signal and the associative signal. Forthis purpose the synapse includes a multiplier block (blocks 1, 8, 12)in which said product is calculated (s*a_(i)). In FIG. 1, signals s anda₁ are applied to the first synapse in order to calculate said product,and the product s*a₁ is calculated in the multiplier block 1. Signals sand a₂ are applied to the second synapse, and the product s*a₂ iscalculated in the multiplier block 8 of the second synapse, and signalss and a_(n) are applied to the nth synapse, and the product s*a_(n) iscalculated in the multiplier block 12 of the nth synapse.

If the signals s and a_(i) can have only the logical values 0 and 1 (forexample, a particular supply voltage value corresponds to the logicalvalue 1 and the zero level corresponds to the logical value 0), thensaid product (s*a_(i)) can be calculated as a logical product. Thelogical product (s*a_(i)) is applied to blocks 2 (first synapse), 9(second synapse) and 13 (n synapse), in which the temporal average ofsaid product is formed and maintained. The temporal average is conveyedfrom blocks 2, 9 and 13 to blocks 3, 10 and 14, respectively.

In blocks 3, 10 and 14 the temporal average is compared to a particularthreshold level. When the temporal average exceeds the threshold level,then said associative signal a_(i) can be considered to correlate withthe main signal s. Thus, a value deviating from zero, i.e. number 1, isset as the actual strength of the synapse or as the weight coefficient,the value otherwise being zero. The desired weight coefficient of thesynapse w_(i), where i=1,2, . . . ,n depending on the synapse concerned,is the output of blocks 3, 10 and 14. For example, the weightcoefficient w₁ of the first synapse is the output of block 3, the weightcoefficient w₂ of the second synapse is the output of block 10, and theweight coefficient w_(n) of the nth synapse is the output of block 14.The weight coefficient of the synapse obtains value 0 or 1.

The weight coefficients of the first, second and nth synapse are appliedto blocks 4, 11 and 15, respectively, in which the weighted productw_(i)* a_(i) is calculated in order to obtain the output signals of thesynapses. Said output signals are applied to block 5, where they aresummed, as mentioned above. The sum is applied to block 6, in which themain signal s is summed with the sum of the output signals obtained fromthe synapses calculated in block 5. The output signal of block 6 isapplied to block 7. Block 7 is a threshold circuit that determines theoutput signal so of the entire neuron by comparing the sum calculated inblock 6 to a particular threshold voltage. The signal so is thus theoutput signal of block 7, which is simultaneously the output signal soof the entire neuron.

A problem with such an associative neuron synapse is the technicalimplementation of the generation and maintenance of the temporal averageof the product of said two signals (s*a_(i)).

FIG. 2 shows a known implementation that is applicable for determiningthe weight coefficient of each associative neuron synapse shown inFIG. 1. This known implementation is a circuit having said two signalsas the input signals thereof, i.e. the main signal s and the associativesignal a_(i). The output signal of the circuit is the weight coefficientw_(i) of the synapse. The subindex i may obtain values 1 to n dependingon which synapse weight coefficient is being determined.

The main signal s and the associative signal a_(i) are applied to an ANDcircuit, AND1, where the logical product of the signals s and a_(i) isformed. The AND1 circuit implements the multiplier block 1, 8 or 12 ofthe synapse shown in FIG. 1. As both the main signal s and theassociative signal a_(i) have a logical value 1, the output of AND1circuit obtains value 1, whereas it otherwise obtains value zero. Asnoted above, the logical value 1 may correspond to a particular voltagevalue of a signal (such as 3.3 V) and the logical value 0 may correspondto the voltage value zero, for example.

The logical product, or the AND1 circuit output, is applied through adiode D1 and a resistor R11 to a temporal averaging RC circuit thatcomprises a resistor R21 and a capacitor C11. The resistor R21 andcapacitor C11 are connected from one end to the ground level and fromthe other end to the other end of the resistor R11. As the logicalproduct is 1, the output of the AND1 circuit is nearly the same as thesupply voltage of the AND circuit, in which case the capacitor C11 ischarged towards the supply voltage with the time constantτ, where τ=R11 * C11

In the time constant equation R11 is the resistance of the resistor R11and C11 is the capacitance of the capacitor C11. The time constantequation holds true when the resistor R21 is much higher than theresistor R11, in which case the. effect of the resistor R21 on the timeconstant can be left unnoticed.

When the logical product is 0, the output of the AND1 circuit goes tothe zero level, whereby the capacitor C11 is discharged (partly orentirely depending on the discharge time) through the resistor R21,until the logical product again obtains the value 1, whereby thecharging state of the capacitor C11 starts to increase again. The diodeD1 prevents the discharging of the capacitor C11 through the resistorR11 and the AND1 circuit. Thus the capacitor C11 can be dischargedthrough the resistor R21 only with the time constantτ, where τ=R21*C11

In the time constant equation R21 is the resistance of the resistor R21and C11 is the capacitance of the capacitor C11.

The capacitor C11 is charged and discharged as a temporal function ofthe logical product s*a_(i). The temporal average of the logical productis then stored in the capacitor and is instantaneously represented by avoltage across the capacitor C11. The diode D1, the resistors R11 andR21 as well as the capacitor C11 thus implement block 2, 9 or 13 shownin FIG. 1.

The voltage across the capacitor C11 is applied to the first input of avoltage comparator COMP1. The input impedance of the voltage comparatorCOMP1 is high so that the capacitor C11 cannot be discharged throughhere. A particular reference voltage V is applied to the second input ofthe voltage comparator COMP1. If the signals s and a_(i) simultaneouslyobtain value 1 sufficiently many times so that the voltage of thecapacitor C11 exceeds the reference voltage of the voltage comparatorCOMP1, i.e. the so-called learning threshold of the synapse, then theoutput voltage of the voltage comparator COMP1 rises up.

The output of the voltage comparator COMP1 is applied to a set resetcircuit SR1. A reset signal generally provided with value 0 is thesecond input in the set reset circuit SR1. The set reset circuit SR1comprises two outputs Q and Q′, one of which being inverted. The valueof the output Q is the weight coefficient w_(i) of the synapse. When theoutput voltage of the voltage comparator COMP1 increases, the set resetcircuit SR1 permanently changes its state from value 0 to value 1, sothat the output Q, i.e. the weight coefficient w_(i) of the synapse,obtains the value 1. The voltage comparator COMP1 and the set resetcircuit SR1 thus implement block 3, 10 or 14 shown in FIG. 1 for thegeneration of the weight coefficient w_(i) of the synapse.

A microcircuit implementation of the RC circuit shown in FIG. 2 presentsa problem, as the required time constants call for high capacitancevalues. However, in practice only fairly small capacitors, withcapacitances of 1 pF or less, can realistically be placed onto themicrocircuit. If the capacitor C11 is of this size, then the resistorR21 must correspondingly have an extensively high value (typically 1 TΩor higher), for the desired time constant. The implementation of such ahigh resistance on the microcircuit is nevertheless difficult. Inaddition, a capacitor of the magnitude 1 pF takes up a lot of space onthe microcircuit.

DISCLOSURE OF INVENTION

The present invention is a new invention that proposes to solve theprior art problem. According to a first aspect of the invention there isprovided a method for determining a weight coefficient of an artificialassociative neuron synapse the weight coefficient being determined bytemporal behaviour of a product of two signals, the method comprising:

forming the product of said two signals and storing the temporalbehaviour of the product of said two signals; and

determining the weight coefficient of the synapse on the basis of thestored temporal behaviour of the product of said two signals.

The method is characterized in that the storing of the temporalbehaviour of the product of said two signals is carried out in a shiftregister chain.

The determination of the weight coefficient of the synapse based on thetemporal behaviour of the product of said two signals refers, forexample, to the determination of the weight coefficient on the basis ofhow frequently said two signals occur simultaneously. Storing thetemporal behaviour of the product of said two signals refers, forexample, to the storage of the values of the product of said two signalsat different moments at a shift register included in the synapse, whichin an embodiment of the invention comprises two or more seriallyconnected D flip-flop circuits. Storing the values of the product ofsaid two signals in the synapse may be temporary in such a manner thatonly a particular number of the product values preceding the presenttime are kept in the shift register at a time.

According to a second aspect of the invention there is provided anartificial associative neuron synapse, whose weight coefficient isdetermined by temporal behaviour of a product of two signals, thesynapse comprising:

a circuit element for forming the product of said two signals andcircuit elements for storing the temporal behaviour of the product ofsaid two signals; and

circuit elements for determining the weight coefficient of the synapseon the basis of the stored temporal behaviour of the product of said twosignals.

The synapse is characterised in that it comprises:

a shift register chain for storing the temporal behaviour of the productof said two signals.

According to a third aspect of the invention there is provided a methodfor determining a weight coefficient of an artificial associative neuronsynapse, wherein the weight coefficient of the synapse is determined onthe basis of temporal average of a product of two signals.

The method is characterized by:

taking temporal samples from the product of said two signals at suchmoments when one of the signals changes so that it deviates from zero,

forwarding said samples into such a shift register chain, from which apredetermined number of said samples taken at previous moments are ateach moment available,

deducing on the basis of said samples taken at previous moments whethera value deviating from zero is to be set as the weight coefficient ofthe synapse.

Preferably but not necessarily a sample is taken at all such timeinstants when one of said signals changes so that it deviates from zero.This means that the sample is taken preferably always (but notnecessarily always), when, for example, the first one of the two signalschanges so that it deviates from zero.

According to a fourth aspect of the invention there is provided anartificial associative neuron synapse, wherein a weight coefficient ofthe synapse is determined on the basis of a temporal average of aproduct of two signals.

The synapse is characterized by comprising:

means for taking a temporal sample of the product of said two signals atsuch moments when one of said signals changes so that it deviates fromzero;

a shift register chain, from which a predetermined number of the samplestaken at previous moments are at each moment available, means forsetting the weight coefficient of the synapse on the basis of thetemporal average of said samples.

It is an object of an embodiment of the invention to bypass therestrictions involved in microcircuit technology by implementing anaveraging circuit as a combination of easily implementable circuitelements that are physically as small as possible.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following the invention is described in greater detail withreference to the accompanying drawings, in which

FIG. 1 illustrates a prior art associative neuron,

FIG. 2 shows a prior art implementation that is applicable fordetermining the weight coefficient of the associative neuron synapseshown in FIG. 1,

FIG. 3 shows a circuit according to a preferred embodiment of theinvention for determining the weight coefficient of an artificialassociative neuron synapse,

FIG. 4 illustrates signal wave forms describing how the circuit shown inFIG. 3 operates, and

FIG. 5 illustrates another implementation of the invention.

BEST MODE FOR CARRYING OUT THE INVENTION

FIGS. 1 and 2 have already been explained in connection with the priorart description. FIG. 3 shows a circuit according to a preferredembodiment of the invention for determining the weight coefficient of anartificial associative neuron synapse.

The input signals of the circuit are a main signal s and an associativesignal a_(i). The output signal of the circuit is a weight coefficientw_(i) of the synapse. The signals s and a_(i) are applied to an ANDcircuit AND1, which forms a logical product (s*a_(i)) of the signals sand a_(i). The logical product is applied to a shift register chain,which is composed of D flip-flop circuits FF1, FF2, FF3 and FF4. Each Dflip-flop circuit comprises an input D, an input for a clock signal >,an output signal Q and an inverted output signal Q′. The D flip-flopcircuits operate in such a manner, that whenever the clock signal isreceived at the input >, that is clocking occurs, the D flip-flopcircuit copies the input signal D to the output Q.

The output of AND1 circuit is connected to the input D of the first Dflip-flop circuit FF1. The output Q of the first D flip-flop circuit FF1is connected to the input D of the second flip-flop circuit FF2 andthrough a resistor R1 to a first input of a voltage comparator COMP1.The output Q of the second D flip-flop circuit FF2 is connected to theinput D of the third D flip-flop circuit FF3 and through a resistor R2to the first input of the voltage comparator COMP1. The output Q of thethird D flip-flop circuit FF3 is connected to the input D of the fourthD flip-flop circuit FF4 and through a resistor R3 to the first input ofthe voltage comparator COMP1. The output Q of the fourth D flip-flopcircuit FF4 is connected through a resistor R4 to the first input of thevoltage comparator COMP1. The first input of the voltage comparatorCOMP1 is connected to the ground level by means of a resistor R0. Asecond input of the voltage comparator COMP1 is connected to a referencevoltage V. The output of the voltage comparator is connected to an inputS of a set reset circuit SR1. A reset signal, which in a standardsituation has value zero, is connected to an input R of the set resetcircuit. The output Q of the set reset circuit is the weight coefficientw_(i) of the synapse. The set reset circuit also includes an invertedoutput Q′. The weight coefficient w_(i) of the synapse is furtherapplied to an AND circuit (not shown in FIG. 3), and the associativesignal a_(i) is applied to the second input thereof. Said AND circuitimplements block 4, 11 or 15 known from FIG. 1, in which a weightedproduct w_(i)*a_(i) is calculated in order to obtain the output signalof the synapse.

All D flip-flop circuits are clocked simultaneously using a clockingsignal. In the example shown in FIG. 3 the associative signal a_(i) isused as the clocking signal. In other words, the associative signala_(i) is applied, in addition to the input of the AND1 circuit, to theinputs > of the clock signal in the D flip-flop circuits via a certaindelay circuit. Buffer circuits B1, B2 and B3 function as the delaycircuit in the example shown in FIG. 3. They cause a slight delay to theassociative signal a_(i) so that the inputs of the D flip-flop circuitshave time to settle before the clocking occurs. Alternatively the mainsignal s can be used for clocking.

Let us next examine in more detail how the circuit shown in FIG. 3operates. At the beginning, all the inputs D and outputs Q of the Dflip-flop circuits are at zero or ground level. The output of the AND1circuit is also at zero. When the main signal s applied to the circuitand the associative signal a_(i) rise to the supply voltage (the supplyvoltage is indicated with symbol U below), that is obtain value 1, theoutput of the AND1 circuit obtains value 1. The output of the AND1circuit is connected to the input D of the first D flip-flop circuitFF1. When the value 1 of the associative signal operating as a clocksignal arrives after a delay to the clocking signal input > of the Dflip-flop circuit FF1, the input D is has already the value 1. Clockingoccurs at the rising edge of the associative signal a_(i) functioning asthe clocking signal, i.e. when the voltage level at the clocking signalinput > rises from zero level to the supply voltage. During thisclocking (referred to as the first clocking), the value 1 in the input Dof the first D flip-flop circuit is copied to the output Q of the firstD flip-flop circuit. In other words, when clocking occurs a temporalsample is taken from the logical product of the signals s and a_(i) andis stored as the output of the first D flip-flop circuit FF1. In theother D flip-flop circuits FF2, FF3 and FF4 the outputs Q still remainat zero value, as the inputs D thereof were set at value zero during thefirst clocking.

When the associative signal a_(i) arriving at the circuit next obtainsvalue 1, the next clocking takes place in the shift register chain. Thisclocking is referred to as the second clocking. Depending on the valueof the main signal s the output of the AND1 circuit and at the same timethe input D of the first D flip-flop circuit now obtain the value 0 or1, which is copied during the second clocking to the output of the firstD flip-flop circuit and to the input of the second D flip-flop circuit.The signal value 1 that was copied during the previous clocking (thefirst clocking) from the input of the first D flip-flop circuit FF1 tothe output, i.e. to the input of the second D flip-flop circuit FF2, isnow copied during the second clocking from the input of the second Dflip-flop circuit FF2 to the output, i.e. to the input of the third Dflip-flop circuit FF3.

During next that is the third clocking the signal at the output of theAND1 circuit, or the signal at the input of the first D flip-flopcircuit FF1 is copied from the input of the first D flip-flop circuitFF1 to the output, i.e. to the input of the second D flip-flop circuitFF2. The signal at the input of the second D flip-flop circuit FF2 iscopied during the third clocking to the output of the second D flip-flopcircuit FF2, i.e. to the input of the third D flip-flop circuit FF3. Andthe signal at the input of the third D flip-flop circuit FF3 is copiedduring the third clocking to the output of the third D flip-flop circuitFF3, i.e. to the input of the fourth D flip-flop circuit FF4.

The voltage signals that indicate the logical value 0 or the logicalvalue 1 thus shift one step forward in the shift register chain duringeach clocking. If the shift register chain comprises four D flip-flopcircuits, then four consecutive values of the logical product s*a_(i)are stored therein, i.e. values(s*a_(i))₀, (s*a_(i))⁻¹, (s*a_(i))⁻², (s*a_(i))⁻³,where the first or the current value is the value calculated during thelast clocking, the second value is the value calculated during theclocking preceding the last clocking, the third value is the valuecalculated two clockings ago and the fourth value is the valuecalculated three clockings ago. Of these logical product values thecurrent value is stored into the output of the first D flip-flop circuitFF1, the value calculated during the clocking preceding the lastclocking is stored into the output of the second D flip-flop circuitFF2, the value calculated two clocking moments ago is stored into theoutput of the third D flip-flop circuit FF3 and the value calculatedthree clockings ago is stored into the output of the fourth D flip-flopcircuit FF4.

The temporal average over the four recent samples of the logical productis formed in the resistor summing network R1, R2, R3, R4, R0, and isobtained from the expression[(s*a_(i))₀+(s*a_(i))⁻¹+(s*a_(i))⁻²+(s*a_(i))⁻³]/4.

The resistor summing network R1, R2, R3, R4, R0 produces a voltagerepresentation of the temporal average of the logical product. Saidvoltage is witnessed at a node point SUM, to which all resistors R1, R2,R3, R4 and R0 are connected from one end. Since the resistors R1, R2,R3, R4 and R0 are in turn connected from one end either to the ground orto the supply voltage U, the voltage V_(SUM) of the node point SUM isobtained from the equation

${V_{SUM} = {\frac{k}{m} \cdot U}},$where k is the number of resistors that are connected to the supplyvoltage U and m is the number of resistors that are connected to theground (zero level). It is assumed, when deducing the equation, that allresistors R1, R2, R3, R4 and R0 are of the same size, meaning that theyhave the same resistance. A typical value for the resistors R1, R2, R3,R4 and R5 is 10 to 100 MΩ, but the resistance of the resistors may alsodeviate from these limits.

It can be observed from the above equation that the voltage V_(SUM) ofthe node point increases linearly, when more and more resistors areconnected to the supply voltage U (the resistor is connected to thesupply voltage U, when the corresponding output of the D flip-flopcircuit has the logical value 1), The voltage V_(SUM) of the node pointSUM is at the same time the input voltage of one input (+ terminal) ofthe voltage comparator COMP1, also called the “first” input above.

The equation above shows that if the number of resistors is 5(m=5) thenthe V_(SUM) may obtain values 0, 1/5 U, 2/5 U, 3/5 U and 4/5 U dependingon how many of the outputs in the D flip-flop circuits FF1 to FF4 havethe logical value 1 (=supply voltage U). If for example two of the fourconsecutive values of the logical product s*a_(i) represent value 1(i.e. m=2) the voltage V_(SUM) of the node point SUM obtains value 2/5U. If in turn all four of the four consecutive values of the logicalproduct s*a_(i) represent value 1, then the voltage V_(SUM) of the nodepoint SUM obtains value 4/5 U.

A reference voltage V is connected to the second input (− terminal) ofthe voltage comparator. The reference voltage V is adjusted in advanceto such a value that if an adequate number of the consecutive productvalues of the signals s and a_(i) obtains value 1 (both signals s anda_(i) obtain value 1), the voltage V_(SUM) of the node point SUM, whichat the same time is the first input signal of the voltage comparatorCOMP1, exceeds the reference voltage V. When the reference voltage V orthe learning threshold of the synapse is exceeded, the output signal ofthe voltage comparator COMP1 rises up to the value 1 causing the setreset circuit SR1 connected to the output of the voltage comparatorCOMP1 to change its state permanently from value 0 to value 1, in asimilar way as described before in connection with the prior artdescription. The output signal of the set reset circuit is the weightcoefficient w_(i) of the associative synapse.

FIG. 4 illustrates signal wave forms describing the operation of thecircuit shown in FIG. 3. The topmost signal is an associative signala_(i), the second signal from the top is the main signal s, the thirdsignal from the top is a summing signal V_(SUM) of the resistor summingnetwork and the lowest signal is the weight coefficient w_(i) of theartificial associative neuron synapse i. The horizontal axis representstime. The vertical axis represents voltage or the logical value.

In the example shown in FIG. 4 the signal wave form of the associativesignal a_(i) and the main signal s is a pulse train. One pulse, whoseheight corresponds to the supply voltage U, corresponds to the logicalvalue 1. The zero level corresponds to the logical value 0. The lengthof the voltage pulse typically ranges from a few milliseconds tohundreds of milliseconds, but it may also vary beyond these limits. Thesignal wave form of the summing signal V_(SUM) of the resistor summingnetwork as a function of time is a staircase function and the signalmode of the weight coefficient w_(i) is a step function.

Calculated from the beginning of the examination at the fourthassociative signal pulse, both the associative signal and the mainsignal obtain value 1. Thus the summing signal V_(SUM) of the resistorsumming network increases by one step, or obtains value 1/5 U. In theexample shown in FIG. 4, both the associative signal and the main signalobtain value 1 at the fifth, sixth and seventh associative signal pulse.The V_(SUM) thus increases by one step again at the fifth pulse, orobtains value 2/5 U. At the sixth pulse the V_(SUM) again increases byone step, or obtains value 3/5 U. Then at the seventh pulse the V_(SUM)increases to voltage value 4/5 U.

In the example shown in FIG. 4 the reference voltage V is set at such avalue that when the V_(SUM) increases to voltage value 4/5 U, theV_(SUM) exceeds the value of the reference voltage, in which case theweight coefficient of the synapse, that is the synaptic weightcoefficient, provided by the set reset circuit permanently rises fromzero to value 1. After this, the values of the associative signal andthe main signal have no effect on the weight coefficient of the synapse.Even though the V_(SUM) drops in the example shown in FIG. 4 during thefollowing associative signal pulses to a level that is lower than thereference voltage V, the weight coefficient w_(i) remains at value 1.

The advantage of the inventive solution shown in FIG. 3 is that it canbe easily realized using basic circuits suitable for microcircuitimplementation. For example, a resistor ranging from 10 MΩ to 100 MΩ isstill of a reasonable size and it can be implemented as an actualresistor or as a FET transistor (Field Effect Transistor) for example.The advantage of the resistor summing method is also the possibility toadjust easily the learning threshold of the synapse by adjusting thereference voltage V of the voltage comparator COMP1. The referencevoltage can for example be set to such a value that the voltage value3/5 U at the + input of the voltage comparator suffices to change theweight coefficient w_(i) from value 0 to value 1.

Applications, in which a fixed learning threshold can be used for thesynapse, may employ the solution shown in FIG. 5 where the resistorsumming network and the voltage comparator COMP1 of FIG. 3 are replacedby combinatory logic.

The input signals a_(i) and s form a logical product in the AND1circuit. The D flip-flop circuits are further clocked at the rising edgeof the associative signal a_(i). The clocking signal is further delayedby buffer circuits B1, B2 and B3, and the value of the logical productis shifted one step forward in the shift register chain during theclocking, as is shown in FIG. 3. The last one of the logical productvalues is found at the output of the first D flip-flop circuit FF1. Thelogical product value at the clocking preceding the last clocking isstored at the output of the second D flip-flop circuit FF2 in the shiftregister chain, the value calculated two clockings ago is stored at theoutput of the third D flip-flop circuit FF3, the value calculated threeclockings ago is stored at the output of the fourth D flip-flop circuitFF4 and the value calculated four clockings ago is stored at the outputof the fifth D flip-flop circuit FF5.

On the circuit shown in FIG. 5, the fixed learning threshold is 4/5,i.e. four cases out of five. In other words, if four out of fiveconsecutive values of the logical product s*a_(i) have value 1, theweight coefficient w_(i) permanently changes from value 0 to value 1.Thus, cases 11110, 11101, 11011 and 10111 are concerned. These cases areidentified using AND circuits AND2, AND3, AND4 and AND5 in the followingway.

The output of the first D flip-flop circuit indicating the last value ofthe logical product (s*a_(i)), or the current value, is conveyed to allAND circuits from AND2 to AND5. In addition the outputs of the second,third and fourth D flip-flop circuit FF2 to FF4 are conveyed to the AND2circuit. Thus the AND2 circuit identifies the case 11110. The outputs ofthe second, third and fifth D flip-flop circuits FF2 to FF3 and FF5 arein addition to the current value of the logical product (s*a_(i))conveyed to the AND3 circuit. The AND3 circuit thus identifies the case11101. In addition to the current value of the logical product (s*a_(i))the outputs of the second, fourth and fifth D flip-flop circuit FF2, FF4and FF5 are conveyed to the AND4 circuit. The AND4 circuit thusidentifies the case 11011. The outputs of the third, fourth and fifth Dflip-flop circuits FF3, FF4 and FF5 are in addition to the current valueof the logical product (s*a_(i)) conveyed to the AND5 circuit. Thus theAND5 circuit identifies the case 10111.

The outputs of the AND circuits AND2, AND3, AND4 and AND5 are connectedto an OR circuit OR1 that identifies the situation in which one of thecases identified by the AND circuits AND2 to AND5 has come true. If oneof the cases identified by the AND circuits has come true, then theoutput of said AND circuit changes to value 1. Since the outputs of theAND circuits are connected to the inputs of the OR1 circuit, the outputof the OR1 circuit simultaneously changes from value 0 to value 1,whenever one of the AND outputs changes to value 1. The output of theOR1 circuit is connected to the input S of the set reset circuit SR1.Thus, as the output of the OR1 circuit changes to value 1, the output ofthe set reset circuit, i.e. the synaptic weight coefficient w_(i)changes permanently from value 0 to value 1.

It should be evident that the solution of the invention can be modifiedwithin the scope of the basic idea of the invention. For example, thelength of the shift register chain can be altered by increasing orreducing the number of D flip-flop circuits. The shift register chainincludes at least two flip-flop type devices, such as D flip-flopcircuits in order that it have the properties of both shifting andmemory. These are also called latches, multivibrators, binaries, etc.Preferably there are more of them, like 4 or 5, but depending on theapplication the number of D flip-flop circuits may be even larger. Theresistors R1, R2, R3, R4 and R0 are preferably of the same size.However, the resistance value of one or more resistors may deviate fromthe other resistance values. For example in certain situations improvednoise tolerance can be achieved if the resistance in the resistor R1 islower than in the other resistors. The resistor summing network can bereplaced by a capacitor network if the input impedance of the comparatorCOMP1 is very high. In that case the resistors R1, R2, R3, R4 and R0 inthe circuit shown in FIG. 3 are replaced with the capacitors C1, C2, C3,C4 and C0 respectively. Preferably the capacitance values of thecapacitors are equal in size. Alternatively the capacitance of one ormore capacitors can, however, deviate from the other capacitance values,and the invention can still be used. The capacitive summing networkdraws less current than a resistive summing network implemented by meansof resistors. In the circuit shown in FIG. 5 the combinatory logic canbe carried out in various ways.

The artificial associative neuron synapse of the invention isimplemented using an appropriate microcircuit technology.

The implementation and embodiments of the invention are described inthis specification by way of examples. It will be evident for thoseskilled in the art that the invention is not restricted to the detailsdescribed in the above embodiments and that the invention can beimplemented in other ways without deviating from the characteristics ofthe invention. The embodiments described are to be consideredinstructive but not restricting. Therefore the implementation and use ofthe invention are only restricted by the attached claims. The differentalternatives to implement the invention defined in the claims includingequivalent implementations are also included in the scope of theinvention.

1. A method for determining a weight coefficient of an artificialassociative neuron synapse the weight coefficient being determined bytemporal behaviour of a product of two signals, the method comprising:forming the product of said two signals and storing the temporalbehaviour of the product of said two signals; and determining the weightcoefficient of the synapse on the basis of the stored temporal behaviourof the product of said two signals, characterized in that the storing ofthe temporal behaviour of the product of said two signals is carried outin a shift register chain.
 2. A method as claimed in claim 1,characterized by taking temporal samples of the product of said twosignals at such moments, when one of said signals changes so that isdeviates from zero, feeding said samples into such a shift registerchain, from which a predetermined number of said samples taken atprevious moments are available at each moment, deducing on the basis ofthe samples taken at the previous moments whether a value deviating fromzero is to be set as the weight coefficient of the synapse.
 3. A methodas claimed in claim 1, characterized in that said two signals and theweight coefficient of the synapse only obtain logical values 0 and 1,and the product of said two signals is formed as a logical product.
 4. Amethod as claimed in claim 1, characterized in that said shift registerchain comprises at least two D flip-flop circuits connected in series.5. A method as claimed in claim 2, characterized in that temporalsamples are taken from said product of two signals always at such amoment when one of said signals changes so that is deviates from zero.6. A method as claimed in claim 2, characterized in that said samplevalues form in a particular summing circuit a sum value, on the basis ofwhich a conclusion is drawn whether a value deviating from zero is to beset as the weight coefficient of the synapse.
 7. A method as claimed inclaim 6, characterized in that said sum value represents a temporalaverage of the product of said two signals.
 8. A method as claimed inclaim 6, characterized in that said sum value is formed using a resistornetwork where all the resistor values are equal.
 9. A method as claimedin claim 6, characterized in that said sum value is formed using aresistor network where at least the value of one resistor deviates fromthe values of the other resistors in the resistor network.
 10. A methodas claimed in claim 8, characterized in that said resistor networkcomprises a set of resistors, which are connected to a common node pointand whereof at least one is connected to the ground level and said sumvalue is formed in said node point.
 11. A method as claimed in claim 6,characterized in that said sum value is formed by means of a capacitornetwork in which all the capacitor values are the same.
 12. A method asclaimed in claim 6, characterized in that said sum value is formed bymeans of a capacitor network, in which the value of at least onecapacitor deviates from the values of the other capacitors in thecapacitor network.
 13. A method as claimed in claim 11, characterized inthat said capacitor network comprises a set of capacitors which areconnected to a common node point and whereof at least one is connectedto the ground level and said sum value is formed in said node point. 14.A method as claimed in claim 2, characterized in that the deduction forsetting the weight coefficient of the synapse to deviate from zero iscarried out using combinatory logic (AND2 to AND5, OR1) which concernssaid sample values.
 15. An artificial associative neuron synapse, havinga weight coefficient determined by temporal behaviour of a product oftwo signals, the synapse comprising: a circuit element (AND 1) forforming the product of said two signals and circuit elements (FF1–FF4)for storing the temporal behaviour of the product of said two signals;and circuit elements (COMP1, SR1) for determining the weight coefficientof the synapse on the basis of the stored temporal behaviour of theproduct of said two signals, characterized in that the synapsecomprises: a shift register chain (FF1 to FF4) for storing the temporalbehaviour of the product of said two signals.
 16. A synapse as claimedin claim 15, characterized by comprising the product of said twosignals, means (AND1, B1 to B3, FF1) for taking a temporal sample of theproduct of said two signals at such a moment when one of said signalschanges so that it deviates from zero, the shift register chain (FF1 toFF4), from which a predetermined number of samples taken at previousmoments are at each moment available, means (R0 to R4, C0 to C4, COMP1,SR1) for setting the weight coefficient of the synapse on the basis ofthe temporal average of said samples.
 17. A synapse as claimed in claim16, characterized by comprising a summing circuit (R0 to R4, C0 to C4)for calculating a particular sum value on the basis of said samples, anda deduction logic (COMP 1) arranged to deduce on the basis of said sumvalue whether a value deviating from zero is to be set as the weightcoefficient of the synapse.
 18. A synapse as claimed in claim 16,characterized by comprising a combinatory logic (AND2 to AND5, OR1)reading said sample values and arranged to deduce whether the weightcoefficient of the synapse is to be set to deviate from zero.
 19. Asynapse as claimed in claim 17, characterized by comprising as saidsumming circuit one of the following: a resistor network (R0 to R4), acapacitor network (C0 to C4).
 20. A method for determining a weightcoefficient of an artificial associative neuron synapse, wherein theweight coefficient of the synapse is determined on the basis of temporalaverage of a product of two signals, characterized by: taking temporalsamples from the product of said two signals at such moments when one ofthe signals changes so that is deviates from zero, forwarding saidsamples into such a shift register chain, from which a predeterminednumber of said samples taken at previous moments are at each momentavailable, deducing on the basis of said samples taken at previousmoments whether a value deviating from zero is to be set as the weightcoefficient of the synapse.
 21. An artificial associative neuronsynapse, wherein a weight coefficient of the synapse is determined onthe basis of a temporal average of a product of two signals,characterized by comprising: means (AND1) for taking a temporal sampleof the product of said two signals at such moments when one of saidsignals changes so that it deviates from zero; a shift register chain(FF1 to FF4), from which a predetermined number of the samples taken atprevious moments are at each moment available, means (R0 to R4, C0 toC4, COMP1, SR1) for setting the weight coefficient of the synapse on thebasis of the temporal average of said samples.